A Note to the Reader

This paper is organized in three almost self contained sections. The first section is a popular discussion on the number of crosswords based on a remark of Shannon made in [Shannon, 1948]. Although never published, the ’back-of-the-envelope’ calculation we present is believed to be the same argument Shannon had in mind when he wrote the passage on crosswords. The second section presents the Hausdorff dimension and the box-counting dimension and gives an introduction to the theory of self similar sets. Methods for calculating the Hausdorff dimension of self similar sets are discussed; in particular we prove Hutchinson’s formula for the dimension of sets represented by iterated function systems (IFS). The third and last section describes how the topics of the two previous sections are in fact connected. Indeed, we argue that the problem of calculating the number of crosswords is related to finding the Hausdorff dimension of certain self similar sets. In addition we identify a connection between box counting dimension and Hartley entropy. I ON THE NUMBER OF CROSSWORDS The American mathematician Claude E. Shannon is widely recognized as the father of information theory. His most famous paper is “A Mathematical Theory of Communication” from 1948 in which he lays the foundation for the modern information theory, and even today the paper shows an impressive combination of clarity and vision. We quote a few passages from the end of section 7, wherein we find a curious reference to suchmundanematters as crossword puzzles. . . The ratio of the entropy of a source to the maximum value it could have while still restricted to the same symbols will be called its relative entropy. This, as will appear later, is the maximum compression possible whenwe encode into the same alphabet. Oneminus the relative entropy is the redundancy. The redundancy of ordinary English, not considering statistical structure over greater distances than about eight letters, is roughly 50%. This means that when we write English half of what we write is determined by the structure of the language and half is chosen freely. The figure 50%was found by several independent methods which all gave results in this neighborhood. One is by calculation of the entropy of the approximations to English. A second method is to delete a certain fraction of the letters and then let someone attempt to restore them. If they can be restored when 50% are deleted the redundancy must be greater than 50%. A third method depends on certain known results in cryptography. Two extremes of redundancy in English prose are represented by Basic English and by James Joyce’s book Finnegans Wake. The Basic English vocabulary is limited to 850 words and the redundancy is very high. This is reflected in the expansion that occurs when a passage is translated into Basic English. Joyce on the other hand enlarges the vocabulary and is alleged to achieve a compression of semantic content.